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ANALYSIS  OF  OFFICER  PERFORMANCE  OF  AN  EXPERIMENTAL  TASK: 
HIGHWAY  TRAFFIC  PLAN 


In  response  to  recommendations  from  the  Army  Scientific  Advisory 
Panel  and  to  needs  of  the  Army  expressed  by  the  Office  of  the  Deputy 
Chief  of  Staff  for  Personnel,  a  major  longitudinal  research  effort  was 
undertaken  to  improve  initial  assignment  of  officers.  Objectives  were 
to  provide  techniques  for  early  identification  of  effective  combat 
leaders  and  to  determine  whether  psychological  measurement  would  be 
effective  in  classifying  officers  differentially  to  the  broad  domains 
of  combat,  technical,  and  administrative  duties. 

Within  the  overall  effort,  4000  officers  were  given  a  two-day 
battery  of  experimental  measures--the  Differential  Officer  Battery 
(DOB)--at  entry  on  active  duty  in  1961  and  1962.  From  this  group  a 
special  sample  of  900  composed  of  approximately  300  officers  assigned 
to  combat  duties,  300  to  technical  duties,  and  300  to  administrative 
duties  was  chosen  to  participate  in  a  special  three-day  performance 
simulation  of  a  field  exercise  at  the  Officer  Evaluation  Center  (0EC) 
established  for  the  purpose  at  Fort  McClellan,  Alabama.  The  exercise 
comprised  15  situational  tasks,  to  be  performed  in  a  simulated  combat 
environment:  5  tasks  designed  specifically  to  represent  the  demands 
of  each  of  the  three  domains-combat,  technical,  and  administrative. 

For  each  task,  a  system  of  objective  records  of  specific  behaviors, 
products,  and  judgments  was  carefully  designed.  Teams  of  officers  and 
enlisted  men  were  trained  to  carry  out  the  exercise  as  actor  participants 
observers.  Recorded  observations,  evaluations,  and  results  were  then 
analyzed  to  yield  dimensions  of  performance  on  each  task.  The  present 
report  describes  findings  from  analysis  of  one  of  the  administrative 
tasks,  the  Highway  Traffic  problem. 


OBJECTIVES  OF  THE  PRESENT  ANALYSIS 

The  objectives  of  analysis  of  the  Highway  Traffic  task  were  to 
identify  the  behavioral  dimensions  measured  and  to  provide  scores  for 
use  in  relating  those  dimensions  to  behavior  in  the  other  14  tasks,  to 
predictor  scores  from  the  DOB,  and  to  evaluations  of  performance  in  the 
real-life  Army  environment.  Findings  of  this  and  parallel  analyses  of 
the  other  tasks  are  to  be  applied  to  indicate  techniques  for  operational 
performance  evaluation  of  junior  officers,  for  early  identification  of 
potential  commanders,  and  for  initial  classification  of  cadets  in  the 
various  officer  training  programs  prior  to  commissioning. 


METHOD  OF  ATTACK 


THE  SITUATIONAL  TASK 

The  Highway  Traffic  problem  was  designed  to  measure  the  resourceful¬ 
ness  and  ingenuity  of  the  examinee  in  accomplishing  a  logistical  support 
mission,  his  ability  to  plan  logistical  support  for  a  large-scale  tactical 
operation  and  to  initiate  plans  and  decisions  in  response  to  rapid  polit- 
cal  and  military  changes,  as  well  as  his  persistance  despite  tedium,  time 
pressure,  harassment,  unforeseen  contingencies,  and  fatigue.  Specifically, 
the  officer  being  tested  was  to  designate  routes  for  movement  of  supplies 
from  rear  areas  to  combat  zones.  He  was  to  extract  and  utilize  informa¬ 
tion  from  maps,  overlays,  and  route  reconnaissance  reports  in  order  to 
develop  a  highway  regulation  plan  making  maximum  use  of  existing  highway 
facilities.  He  was  to  select  locations  for  control  points  to  regulate 
traffic. 


SAMPLE 

The  full  sample  consisted  of  approximately  900  officers  who  attended 
the  OEC.  For  the  internal  analysis  presented  here,  however,  only  the 
last  732  cases  were  used  because  minor  changes  and  additions  in  record¬ 
ing  and  evaluation  procedures  were  made  after  earlier  cases  had  gone 
through  the  Center.  These  changes  are  not  sufficient  to  preclude  apply¬ 
ing  certain  scores  derived  here  to  the  earlier  cases  for  the  validation 
of  the  DOB. 


VARIABLES 

Two  highway  traffic  plan  problems  were  presented,  each  requiring  a 
plan  to  deliver  specific  tonnages  of  Class  I  (subsistence  supplies. 

Class  III  (petroleum  fuel  and  products),  and  Class  V  (ammunition) 
supplies  from  available  depots  to  specified  forward  supply  points  over 
a  road  network  of  which  sections  had  been  destroyed  or  damaged  by  enemy 
action.  The  destruction  included  complete  interdiction  of  certain  routes 
due  to  radiation  from  nuclear  explosions.  There  were  six  scores  for 
tonnage  delivered — Class  I,  III,  and  V  in  each  of  two  problems.  In 
scoring,  credit  was  given  for  delivery  of  the  required  tonnage  by  class 
to  the  forward  supply  points.  Additional  credit  for  minimizing  the  dis¬ 
tance  traversed  (a  ton-kilometer  product)  was  given  only  if  all  required 
tonnage  was  delivered.  Tonnage  routed  over  interdiction  routes  or  in 
excess  of  a  route's  capacity  was  considered  not  to  have  been  delivered. 

Performance  variables  consisted  of  the  highway  traffic  plans  pro¬ 
duced  as  recorded  by  observers  on  the  Highway  Traffic  Plan  Worksheet  and 
evaluations  of  examinee  behavior  by  observers  on  the  Highway  Traffic  Plan 
Checklist,  the  Rating  Supplement,  and  the  Descriptive  Report.  These  in¬ 
struments  and  the  44  variables  derived  from  them  are  shown  in  Appendix 
Table  A-l.  The  objective  products  (the  plans  produced)  were  scored 
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separately.  The  behaviors  were  observed  and  evaluated  across  traffic 
plans.  A  minimum  of  two  raters  is  represented  on  all  ratings  used. 
Ratings  were  made  independently  and  raters  met  to  resolve  differences 
to  produce  agreement  on  the  evaluation.  Table  1  categorizes  the  44 
performance  variables  selected  for  analysis  from  the  observations  and 
evaluations  recorded. 


Table  1 

PERFORMANCE  VARIABLES  OF  HIGHWAY  TRAFFIC  PLAN  TASK 


Instrument 

Type  of  Variable 

No .  of 
Variables 

Highway  Traffic  Plan 

Worksheet 

Tons  delivered--3  classes 
for  each  2  plans 

6 

Ton-kilometer  products--l 
for  each  plan 

2 

Highway  Traffic  Plan 

Checklist* 

Necessary  provisions  of  plan 

23 

Descriptive  Report* 

Aspects  of  task  performance 

10 

Rating  Supplement* 

Motivation 

1 

Attitude 

1 

Understanding  mission 

1 

Total 

44 

a  A  single  score  on  each  of  the  behavior  variables  was  obtained  across  the 
two  problems  presented. 


Population  control  variables  were  broad  category  of  assignment 
(combat,  technical,  administrative),  component  (Regular  Army  or  Reserve), 
and  grade  (first  lieutenant  or  second  lieutenant).  Analysis  with  respect 
to  these  variables  will  be  presented  in  a  separate  report  covering  other 
of  the  situational  problems  forming  the  simulated  field  exercise. 

A  factor  analysis  (in  two  steps)  was  performed  as  a  basis  for  identi¬ 
fying  the  behavioral  dimensions  represented.  The  first  step  was  to  obtain 
distributions  of  the  performance  variables.  On  the  basis  of  these  dis¬ 
tributions,  the  tonnage  delivered  and  ton-kilometer  product  scores  were 
coded  in  a  single  scale,  yielding  an  objective  score  for  each  of  the  two 
problems.  The  23  variables  of  the  Highway  Traffic  Plan  Checklist  were 
correlated  with  scores  on  motivation,  attitude,  and  understanding  mission. 
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The  resulting  matrix  of  26  variables  was  then  factor-analyzed  by  the 
principal  axes  method  using  the  highest  off-diagonal  entry  as  the  commu- 
nality  estimate  at  each  step.  The  factor  matrix  was  rotated  by  the 
varimax  method  and  eight  factors  were  identified.  Scales  of  items  repre¬ 
senting  the  factors  were  derived;  these  scales  were  then  correlated  in  a 
new  matrix  with  scores  derived  from  the  Descriptive  Report  and  the  Highway 
Traffic  Plan  Worksheet.  Factoring  and  rotation  of  this  new  matrix  of  17 
variables  yielded  the  final  scales  and  total  score  for  the  situational 
task.  These  scales  and  total  score  included  all  the  original  variables 
except  those  with  extreme  p-values. 


RESULTS 


CODING  OF  OBJECTIVES  SCORES 

Distributions  of  the  six  scores  for  tonnage  delivered  proved  to  be 
U-shaped,  56$  to  87$  of  the  sample  achieving  full  delivery  and  11$  to  29$ 
achieving  little  or  none.  Table  2  shows  the  percentage  achieving  full, 
part,  or  negligible  delivery.  As  a  result  of  these  findings,  the  raw 
tonnage  scores  were  converted  to  a  coded  score,  9  being  awarded  for  full 
delivery,  4  to  2  for  partial  delivery,  and  0  for  little  or  none. 

For  those  who  achieved  full  delivery  of  all  classes  of  supply  in  a 
given  problem,  additional  credits  were  awarded  for  efficiency  of  delivery, 
i.e.,  the  smaller  the  total  of  ton-kilometer  products,  the  greater  the 
credit.  Table  3  shows  the  percent  of  the  sample  who  received  additional 
credit  up  to  9  points.  Thus,  it  was  possible  to  achieve  a  maximum  score 
of  36  points  on  each  problem:  27  for  delivery  of  the  full  tonnage  re¬ 
quirement  and  9  additional  points  for  efficiency  of  delivery.  The  coded 
total  score  for  each  problem  and  the  grand  total  were  Included  in  further 
analysis. 


Table  2 

PERCENT  OF  SAMPLE  ACHIEVING  DELIVERY  OF 
FULL,  PART,  OR  NONE  OF  REQUIRED  TONNAGE 


Requirement  Met 

Class 
Prob  1 

I 

Prob  2 

Class 
Prob  1 

III 

Prob  2 

Class 
Prob  1 

V 

Prob  2 

Full 

69 

87 

TO 

83 

56 

82 

Part 

12 

2 

16 

6 

15 

6 

Little  or  none 

19 

11 

14 

11 

29 

12 

Table  3 


PERCENT  OF  SAMPLE  ACHIEVING  ADDITIONAL  CREDIT 
FOR  EFFICIENCY  OF  DELIVERING  REQUIRED  TONNAGE 


Credit  Received 

Problem  1 

Problem  2 

1-9 

9 

7 

4-6 

18 

36 

1-3 

20 

28 

0  (Full  delivery) 

2 

4 

0  (Part  delivery) 

45 

19 

0  (Little-no  delivery) 

6 

6 

FACTOR  ANALYSIS  OF  TRAFFIC  PLAN  CHECKLIST 

In  the  factor  analysis  of  observed  behavior,  26  variables  were 
included — 23  aspects  of  the  plan  covered  in  the  Checklist  and  the  three 
additional  rating  variables,  motivation,  attitude,  and  understanding  the 
mission.  Eleven  orthogonal  factors  were  rotated  to  simple  structure  by 
the  varimax  method.  The  8- factor  solution  was  selected  as  offering  the 
optimum  level  of  differentiation  and  factor  definition.  Table  4  presents 
the  items  selected  to  represent  each  factor.  Two  factors  represent 
overall  performance  on  the  problems,  five  represent  specific  aspects 
across  problems,  and  one  is  concerned  with  effective  motivation.  The 
complete  matrix  is  given  in  Table  A-l. 


FACTOR  ANALYSIS  OF  TOTAL  TASK  PERFORMANCE 

Analysis  of  a  matrix  of  the  eight  factor  scales,  the  three  objective 
scores,  and  six  items1-'  from  the  Descriptive  Report  yielded  16  factors. 
Means  and  standard  deviations  appear  in  Table  A-2.  Of  these  factors,  how¬ 
ever,  only  the  first  six  were  retained.  Table  5  presents  the  variables 
and  loadings  on  these  factors.  Only  loadings  over  .30  are  given. 


I' Four  items  were  dropped  because  of  insufficient  variance — p  values 
beyond  10  or  90 . 
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FACTOR  LOADINGS  OF  OBSERVED  BEHAVIOR  AND  MOTIVATION  VARIABLES 
IN  HIGttlAY  TRAFFIC  PLAN  TASK 
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Table  5 


FACTOR  LOADINGS  OF  PERFORMANCE  VARIABLES 
IN  HIGHWAY  TRAFFIC  PLAN  TASK 


Factor/Items 

Loadings 

Scarce 

I. 

Problem  Solving 

Coded  total  objective  score 

•95 

HT  Plan  Worksheet 

Coded  score, problem  2 

.84 

HT  Plan  Worksheet 

Tonnage  factor 

•77 

HT  Plan  Checklist 

Coded  score, problem  1 

•  75 

HT  Plan  Worksheet 

Mission  accomplishment 

•  51 

Descriptive  Report 

Following  instructions 

.45 

Descriptive  Report 

II. 

Response  to  Stress 

Endurance  and  stamina 

.76 

Descriptive  Report 

Cool,  level-headed 

.68 

Descriptive  Report 

III. 

Procedural  Skills 

Designating  routes  factor 

.72 

HT  Plan  Checklist 

Designating  controls  factor 

.60 

HT  Plan  Checklist 

Performance  factor  (prob.  2) 

.60 

HT  Plan  Checklist 

Performance  factor  (prob.  1) 

.54 

HT  Plan  Checklist 

IV. 

Attention  to  Instructions 

Noting  route  restrictions  factor 

.72 

HT  Plan  Checklist 

Following  instructions 

.60 

Descriptive  Report 

Mission  accomplishment 

•52 

Descriptive  Report 

Coded  score,  problem  1 

•  36 

HT  Plan  Worksheet 

V. 

Identifying  Depots 

Identifying  depots  factor 

•74 

HT  Plan  Checklist 

Noting  route  restrictions  factor 

.42 

HT  Plan  Checklist 

Bearing  and  assurance 

.31 

Descriptive  Report 

VI. 

Style  of  Performance 

Effective  expression 

.78 

Descriptive  Report 

Bearing  and  assurance 

.61 

Descriptive  Report 

Effective  motivation  factor 

•57 

Rating  Supplement 

Factor  I  clearly  defined  the  final  product  of  the  task:  objective 
tonnage  delivery  score  and  rated  effectiveness  in  carrying  out  the  mis¬ 
sion.  Factor  II  refers  to  effective  functioning  under  pressure.  Com¬ 
pletion  of  the  important  aspects  of  the  task,  or  procedural  skill,  com¬ 
prises  Factor  III.  Although  Factors  IV  and  V  are  differentiable,  they 
both  appear  to  be  concerned  with  attending  to  instructions  and  situational 
data,  while  Factor  VI  clearly  reflects  a  confident  style  of  performance. 
Loadings  on  the  factors  are  given  in  Table  A-3- 

Based  on  these  results,  the  following  composite  scores  were  identi¬ 
fied  to  be  used  in  further  analysis  across  the  full  15  situational  tasks: 

Coded  Total  Objective  Score  (first  variable  on  I) 

Response  to  Stress  (both  variables  on  II) 

Procedural  Skills  (four  variables  on  III) 

Attention  to  Situational  Requirements 
(first  three  variables  on  IV  plus  first  on  V) 

Style  of  Performance  (three  variables  on  VI) 


SUMMARY  AND  CONCLUSIONS 

Factor  analysis  of  relationships  among  the  variables  of  the  Highway 
Traffic  Task--the  objectively  scored  traffic  plan,  the  observers'  check¬ 
list  of  the  plan's  provisions,  and  ratings  of  aspects  of  the  officer's 
performance,  including  attitude  and  motivation--yielded  five  major  per¬ 
formance  factors:  These  were  l)  total  objective  score,  2)  response  to 
stress,  3)  procedural  skills,  4)  attention  to  requirements,  and  5)  style 
of  performance.  These  scores,  together  with  certain  single  variables  of 
interest,  are  to  be  analyzed  in  a  matrix  including  scores  derived  from 
the  other  14  situational  tasks.  Results  are  to  be  applied  to  problems 
of  officer  performance  evaluation  and  early  identification  of  the  most 
promising  officer  leaders. 
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APPENDIX.  DETAILED  RESULTS  OF  ANALYSES  -  HIGHWAY  TRAFFIC  PLAN  TASK 


Table  A-l 

FACTOR  MATRIX  OF  OBSERVED  BEHAVIOR  AND  MOTIVATION  VARIABLES 


Problem 

Item 

I 

II 

III 

Factor  Loadings 

IV  V  VI  VII 

VIII 

h2 

1 

Identified  depots 

02 

98 

02 

0  b 

04 

04 

06 

it 

2 

Identified  depots 

00 

06 

-01 

02 

06 

02 

Oft 

15 

1 

Used  direct  routes 

36 

06 

00 

18 

-07 

20 

17 

T  * 

07 

o 

c 

Used  direct  routes 

00 

13 

°5 

11 

-01 

59 

17 

O  ~z 

C-  J 

46 

1 

Used  closest  depots 

48 

15 

03 

05 

12 

20 

34 

21 

4  6 

2 

Used  closest  depots 

02 

07 

02 

-23 

01 

40 

17 

cr 

25 

1 

Desig.  control  points 

60 

19 

10 

-04 

61 

26 

00 

06 

-4 

2 

Desig.  control  points 

37 

11 

19 

05 

66 

65 

02 

02 

1  .04* 

1 

Desig.  reserve  MSR 

-09 

06 

-04 

02 

78 

-17 

16 

If 

yr, 

2 

Desig.  reserve  MSR 

-06 

-03 

13 

08 

83 

05 

04 

12 

74 

1 

Refrained  from  alt.  routes 

94 

C'~ 

O 

1 

52 

-07 

23 

-03 

40 

-11 

~r 

2 

Refrained  from  alt.  routes 

-14 

-06 

99 

11 

17 

46 

33 

11 

1 . 37 4 

1 

Noted  route  restrictions 

09 

06 

10 

■3 

03 

0c 

09 

17 

2 

Noted  route  restrictions 

06 

07 

-04 

82 

06 

0- 

09 

0  ■ 

72 

1 

Desig.  travel  direction 

59 

-17 

01 

-02 

-05 

37 

21 

24 

62 

2 

Desig.  travel  direction 

20 

-06 

11 

04 

-08 

68 

21 

20 

61 

1 

Desig.  route  tonnage 

37 

02 

10 

-02 

-01 

05 

80 

18 

82 

2 

Desig.  route  tonnage 

02 

03 

17 

-04 

09 

43 

70 

24 

78 

1 

Correct  tonnage  deliv. 

17 

11 

-04 

19 

11 

23 

65 

07 

57 

2 

Correct  tonnage  deliv. 

01 

05 

-03 

17 

21 

38 

52 

04 

50 

1 

Used  routes  on  hwy  capabil . 

50 

00 

-01 

04 

-04 

-06 

02 

06 

26 

2 

Used  routes  on  hwy  capabil . 

11 

-14 

05 

33 

06 

77 

13 

10 

77 

1 

Reserved  march  route 

08 

09 

88 

00 

04 

01 

-11 

05 

oO 

Understanding  mission 

05 

22 

10 

07 

17 

10 

15 

62 

52 

Motivation  rating 

23 

11 

01 

18 

08 

27 

19 

85 

‘‘4 

Attitude  rating 

22 

10 

-02 

13 

10 

25 

08 

88 

42 

*  Loadings  above  1.00  are  believed  to  be  due  to  use  of  tetrachoric  correlations 
or  to  the  fact  that  communal it ies  were  not  reestablished. 
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Table  A-2 


MEANS  AND  STANDARD  DEVIATIONS  OF  FINAL  VARIABLES 
SELECTED  IN  HIGHWAY  TRAFFIC  PLAN  TASK 


Variable 

M 

SD 

Problem  1  solution 

21.11 

11 .09 

Problem  2  solution 

26.11 

).50 

Sum  of  2  solutions 

47  .22 

17  .16 

Bearing  and  assurance 

1.11 

•57 

Effective  expression 

1.02 

•34 

Cool,  level-headedness 

1.04 

.29 

Endurance  and  stamina 

1.03 

.40 

Following  instructions 

.78 

.88 

Mission  accomplishment 

•73 

.86 

Problem  1  performance 

4.43 

1.02 

Problem  2  performance 

4.38 

1.12 

Identifying  depots 

1.23 

•95 

Designating  routes 

2.21 

•55 

Noting  restrictions 

1.63 

.46 

Designating  controls 

2.86 

1.11 

Specifying  tonnage 

2 .99 

1.19 

Effective  motivation 

9. 61 

2.22 
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Table  A-3 


FACTOR  MATRIX  OF  FINAL  VARIABLES  SELECTED 


IN 

HIGHWAY 

TRAFFIC 

PLAN 

TASK 

Variable 

I 

II 

Factor*  Loadings 
III  IV  V 

VI 

h2 

Problem  1  solution 

75 

04 

02 

36 

-08 

11 

71 

Problem  2  solution 

84 

01 

12 

-16 

22 

06 

80 

Sum  of  2  solutions 

95 

03 

08 

14 

07 

10 

94 

Bearing  and  assurance 

12 

23 

14 

07 

31 

61 

56 

Effective  expression 

11 

-03 

10 

02 

-04 

78 

64 

Cool,  level-headedness 

-01 

68 

-02 

08 

00 

21 

51 

Endurance  and  stamina 

12 

76 

12 

-04 

09 

-03 

61 

Following  instructions 

45 

17 

15 

60 

-17 

17 

67 

Mission  accomplishment 

51 

25 

10 

52 

-16 

20 

67 

Problem  1  performance 

22 

06 

54 

30 

-18 

31 

57 

Problem  2  performance 

22 

13 

60 

08 

-11 

29 

52 

Identifying  depots 

10 

10 

03 

06 

74 

16 

60 

Designating  routes 

01 

15 

72 

-02 

18 

-22 

62 

Noting  restrictions 

-02 

-10 

09 

72 

41 

02 

71 

Designating  controls 

10 

-li 

60 

04 

06 

23 

45 

Specifying  tonnage 

77 

07 

23 

06 

06 

19 

69 

Effective  motivation 

25 

24 

15 

22 

15 

57 

54 

*  The  six  factors  accounted  for  63.73$  of  the  total  variance. 
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